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(54) Tide: DIGITAL VIDEO LAN SYSTEM 



(57) Abstract 



This invention discloses a digital 
video system for use in a LAN. The sys- 
tem comprises a video digitizer for receiv- 
ing and digitizing a plurality of analog 
video inputs. A mixer receives the plural- 
ity of digitized video signals for spatially 
mixing the signals in real time into a sin- 
gle frame of a predetermined n by m pixel 
images. The digitized images are coupled 
to a video compressor for compressing 
each of the single frame images transmit- 
ting the images on a LAN to a receiver. 
The receiver decompresses the transmit- 
ted images for near real time display. In 
a preferred embodiment, the compression 
used is wavelet compression. 
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DIGITAL VIDEO LAN SYSTEM 



The present invention relates to a video monitoring and display systems for use in a 
LAN environment, and more particularly to a digital video mixer and compressor system. 

5 

BACKGROUND OF THE INVENTION 

It is well known to provide video security systems in which video cameras are used to 
generate video signals representative of locations for which security surveillance is desired. 
In a typical system, some or all of the video signals are displayed on separate video screens 
10 for monitoring by security personnel. It is also known to record some or all of the video 

signals on video tape, either to provide evidentiary support for the observations by security 
personnel or in cases where "real-time" human monitoring of the signals is impractical or is 
not desired. 

However, video tape suffers from serious drawbacks as a storage medium, 

15 particularly in view of the large quantity of video information generated by video security 
systems. A major concern is the sheer quantity of tapes to be stored, especially when it is 
desired to record signals generated by a large number of surveillance cameras. Moreover, in a 
large system many video tape recorders may be required, resulting in a large capital 
expenditure, and also the need for allocate space for the recorders. Another problem is the 

20 need to frequently change tape cassettes. 

In United States Patent 4,943,854 a Video surveillance system for selectively 
selecting processing and displaying the outputs of a plurality of TV cameras is described. 
The multi-video recorder incorporates a multiplexing means permitting selection of the video 
signal from each of a plurality of TV cameras according to a predetermined importance or 

25 priority, and a priority changed due to occurrence of a new event and transmission of the 
video image thus selected over a single line and recording it by a single video tape recorder 
by time shared multiplexing the video signal at every frame, field or unit of time. It also can 
reproduce or demultiplex the video signal recorded as multiplexed, with each designated TV 
camera or by sequentially selecting the video signal from each TV camera. The system also 

30 has a means for inserting in the time shared multiplexed video signal and control signal such 
as select information, event information, time information or the like, so it can selectively 
demultiplex the video signals recorded as multiplexed, as desired. 

1 
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In order to display full motion video, a frame rate of 30 frames per second (fps) is 
normally required. A limitation of the time-shared multiplexed systems is the reduced frame 
rate and hence choppy picture. For example, for four video inputs, a multiplexed system will 
have a reduced frame rate of approximately 7.5 frames per second. Furthermore, in current 
5 systems if groups of cameras are located a relatively remote area, the video signal has to be 
fed back to a central recording or monitoring station. This requires multiple coaxial cables 
for video transmission, or even if a multiplexing technique is used, a single long length of 
coaxial cable is required. Long cable runs require line amplifiers. In both cases, the costs are 
substantial. 

10 A further disadvantage of current systems is that the images are recorded as analog 

signals on VHS or similar tapes, making archiving cumbersome. 

Thus there is a need for a system that reduces the reliance on coaxial cable for video 
transmission, allows a distributed system architecture instead of linking all cameras 
individually, hence saving in cabling and line amplifier costs. Furthermore such a system 

15 should provide digital video or image recording on a computer mass storage device or the 
like , including being able to run under software control for providing flexibility of 
controlling image routing or viewing, recording, playback and archiving. 

Current systems do not provide live speed and quality across multiple channels nor do 
they provide flexibility of a large number of cameras without losing frames. 

20 

SUMMARY OF THE INVENTION 

The present invention seeks to provide a solution to the problem of combining 
multiple video inputs into a single display without a significant loss of frames. 

In accordance with this invention there is provided a digital video system comprising: 
25 a) a video mixer for spatially mixing a plurality of analog video input signals in 

real time and compressing said mixed signals for transmission; and 
b) a receiver for receiving and decompressing said compressed signals for 

display in near real time. 
In accordance with a further embodiment of the invention there is provided a plurality 
30 of video mixers coupled to a network and a transmitter for receiving the compressed images 
from the plurality of transmitters. 
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In accordance with a still further embodiment there is provided a video system 
comprising: 

a) a video digitizer for receiving and digitizing a plurality of analog video inputs; 

b) a mixer for receiving said plurality of digitized video signals and spatially 
mixing said signals in real time into a single frame of a predetermined n by m 
pixel image; 

c) a video compressor for compressing each said single frame images; 

d) a transmitter for transmitting said images; and 

e) a receiver for receiving and decompressing said transmitted images. 

BRIEF DESCRIPTION OF THE DRAWINGS 

A better understanding of the invention will be obtained by reference to the detailed 
description below in conjunction with the following drawings in which: 

Figure 1 is a block diagram of a video transmitter subsystem according to an 
1 5 embodiment of the invention; 

Figure 2 is a schematic diagram of a singe frame; 

Figure 3 is a software flow diagram for the video transmitter system; 

Figure 4 is a schematic diagram of digital video system according to the present 
invention; and 

20 Figures 5(a) - (t) illustrate various screen shots of a user interface. 



DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 

Referring to Figure 1 a video transmitter subsystem according to an embodiment of 
the invention is shown generally by numeral 1 . The video subsystem 1 is a standalone unit, 
25 which is capable of spatially mixing four analog NTSC or PAL video inputs in real-time into 
a single frame of 640 x 480 pixels image. Each input image occupies a quadrant of the frame 
60 as shown schematically in figure 2. The video subsystem 1 compresses each image frame 
60 using wavelet compression based hardware for a programmable compression ration of 4:1 
to 350:1 . An advantage of wavelet compression is the blocky artifacts, commonly appearing 

30 
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in JPEG type compression, are totally eliminated. The resulting wavelet compressed image 
data is then transmitted via a (100MB or better) Ethernet network in which plurality of video 
transmitters may be joined together. 

In general, the video mixer is considered to have two main sections. First, is a 

5 computer (running QNX operating system) and second, is a PCI PC plug-in board which 

processes the incoming video images and performs wavelet compression. A number of video 
mixers may be installed in a LAN hosted for example by an NT *™ server. The server is 
responsible for retrieving and storing images obtained from the mixers. In this way software, 
updates to the video mixers may be performed from the central server or from the first mixer, 

1 0 thereby minimizing on-site maintenance of each mixer. The computer may be a PC 

motherboard of a baby AT type. The computer should include sufficient memory, hard disk 
space and power source to run the operating system and video transmitter application 
program (discussed below). This type of computer including a PCI interface and network 
adapter is well known in the art and will not be discussed further. 

1 5 The video mixer 1 comprises an analog front end 4 having four independent video 

inputs A, B, C and D coupled to respective anti-alias passive circuits for lightly low-pass 
filtering the video inputs which are then fed into respective text overlay chips (not shown) 
before being coupled to respective video decoders 8 labeled DecoderA, DecoderB, DecoderC 
and DecoderD respectively. In a preferred embodiment, the decoders are a Samsung KS0122 

20 multistandard video decoder and the text overlay chip is a uPD6465. Details of these devices 
including application notes are available in their respective data sheets incorporated herein by 
reference. Based on the above decoder device, the composite video signals are digitized into 
4:2:2 8-bit YcrCb data format at a digitization frequency of 12.27 MHz by the decoders 8. 
The digitization is selected for a square pixel type and therefore the spatial resolution is 

25 640x480 pixels per field. 

The digitized outputs are coupled to a time base corrected spatial mixer 20, which 
comprises a plurality of FIFO registers 24 for storage of pixel data from the analog front end; 
a time base corrector 26 and a quadruple retrieval controller (QRC) 28. The spatial mixer 20 
assembles a 640x480 pixel frame which is comprised of four horizontally decimated image 

30 fields (of size 320x240) from each of the digitized outputs. 

Each of the four digitized channels is coupled to a dedicated bank of the FIFO's such 
that each bank stores at least 320x240 pixels. The FIFO banks in each of the channels are 
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coupled to a common output video bus and are individually addressable such that any of the 
320x240 fields from any of the channels may be retrieved. In a preferred embodiment, each 
FIFO register bank consists of two uPD42280 FIFO chips. Each chip is capable of holding 
one 320x240 field of pixel data. Details of these devices including application notes are 
5 available in their data sheets incorporated herein by reference. 

The pixel data retrieval is asynchronous to any of the input video timing, thus the time 
base corrector 26 is coupled to the FIFO's and the decoders to control the retrieval process 
when the input video timing and the retrieved data timing drift with respect to each other. 

The retrieved data from the FIFO's are coupled to a wavelet compression engine 40, 

1 0 which is implemented in an Analog Devices multi format video Codec chip, part number 
ADV60L The ADV601 accepts YcrCb pixel data from its video interface and performs 
wavelet compression thereon. The compressed data is output to a host bus interface 44. 
Conversely the chip may also perform decompression of data received on its host bus 
interface. The ADV601 's host interface is a 32-bit wide interface and consists of four 32-bit 

15 registers. These registers are I/O mapped onto a PCI host bus. Compressed data can thus be 
transferred using PCI burst I/O when an interrupt is generated by data being available in the 
ADV601 internal FIFO. 

Since the wavelet compression engine can take video on a field by field (i.e. 640x240) 
basis, the quadruple image frame must be fed into the engine in two halves - upper and lower 

20 half. The retrieval algorithm for implementing this is implemented in the QRC. It may be 
noted that the spatial mixer may also be bypassed in order to provide full 640x480 fields to 
the wavelet engine for compression. As described above, the quadruple image mixer is made 
up of four banks of field FIFOs (uPD42280). Each decoder channel has a dedicated FIFO 
bank which is capable of storing two quarter size fields of video. The incoming digitized 

25 video is written into the FIFO bank using decoder timing. Since the stored video is of quarter 
size, the video data from the decoder is decimated first (by the decoder). Each bank of stored 
video is read in sequence to construct a quadruple image frame of sixw 640x480. The tope 
two images are retrieved as field- 1 input to the wavelet engine while the bottom two images 
are retrieved as field-2. The reason that two banks of FIFO are needed for each decoder 

30 channel is that TBC function is implemented to avoid read and write collision caused by 
asynchronous timing between the input and ADV601 video timing. The TBC function is 
implemented in the firmware. 
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For single source compression, this quadruple mixer is bypassed. The video data and 
video sync signals are fed directly into the wavelet engine which is placed in slave mode. 
Write control signals are generated by an Altera chip, preferably 6016which is capable of 
taking four asynchronous clocks from each decoder. 
5 Retrieval logic signals, ADV601 syncs and read control signals are generated by 

another Altera chip. 

In quadruple mode,ADV601 is placed in master mode where all the video sync 
signals are generated. In single source mode, the sync signals are provided by the video 
decoders. 

10 During compression the ADV601 requires external numeric processing on the 

statistics of the current field data. This means that an embedded computer will evaluate the 
bin-width values based on the statistics of the field data. The bin-width statistics are 
transferred via the ADV601 host interface and then the PCI interface to or from the 
embedded computer. 

1 5 The PCI interface consists of a device, which is capable of bridging the on-board 

devices onto the PCI bus. It manages the I/O and memory mapping, read/write cycles and 
DMA cycles. It should be capable of bursting data at a rate of 20-30 MB per second. 
Typically the PCI interface is implemented by a PLX PCI9080 PCI accelerator chip. 

A generals purpose 8-bit I/O port is provided to the PCI controller and is configured 

20 as an 4 input 4 output bits. It is mapped on the PCI bus i/o, and is thus under the control of 
the embedded computer. This port may be used to control alarm equipment. 

In a preferred embodiment, a video encoder 50 is provided for generating composite 
and S-video output signals for monitoring or for playback of the decompressed video frame. 
The encoder is a Brooktree BT866 device that is configured in a slave mode and is coupled to 

25 the timing from the ADV601 chip. Data for the encoder is tapped off the data stream to the 
aADV601 video bus. This data is non-interlaced, therefore a field FIFO register 52 is needed 
to convert the data back to interlaced video before it is fed to the encoder. The FIFO registers 
52 are implemented in four uPD42280 devices used to hold two fields of pixel data. 

Referring now to figure 3, a software flow diagram for the video mixer 1 is shown 

30 generally by numeral 100. The software in general is centered on the transmission of 

compressed image data to the system host via the Ethernet LAN. The software comprises an 
application program 1 16 for executing high level commands from a remote host. This 
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includes video channel switching, alarm ports reporting, transmission of wavelet data to the 
remote host when requested, Bin-width calculation, track bum-in time/date, general 
maintenance of the connection to the network and self diagnostics when requested by the 
host. The software also includes a number of low-level drivers and libraries. 
5 A device driver module 1 02 is provided for the PCI 9080 chip for communication 

between the video mixer and the computer. The driver 102 includes low-level services for 
handling PCI bus i/o, DMA and IRQ. It contains functions for translation between physical 
and linear memory, interrupt dispatcher and bus master control. Detailed implementation of 
these functions is well known in the art and will be not discussed further. The software also 

10 includes a I2C bus communication module 104 for obtaining data packets from the 

application and translating data into signal streams. A video control module 106 is provided 
for controlling the video hardware including decoder, encoder and field FIFO. The functions 
implemented by this module include selection of video capture mode, control of contrast and 
brightness and initialization of video chips, NTSC/PAL selection and such like. A TCP/IP 

1 5 managerl 08 is implemented as a socket for handling command and compressed data 

transmission to the host. It calls a QNX LAN driverl 10 to perform the data transmission. A 
wavelet control module 1 12 implements functions to control the data transfer between the 
ADV601 and the host memory, compression ratio, ADV601 initialization and such like. A 
Time/Date port control module 1 14 is provided to load a desired burn-in time/date on each 

20 selected video channel. 

Referring to figure 4, a schematic diagram of a digital video system is shown 
generally by numeral 400. The system comprises a plurality of video mixers 1 including their 
respective computer coupled to a host computer 404 via an Ethernet LAN 406. The host 
computer 404 includes a software-based receiver, which communicates with the multiple 

25 digital video transmitters 1 on the Ethernet network to receive compressed digital images. 

The received images are decompressed in software and displayed 410 in near real time (12-15 
frames per second) and can be recorded to a hard disk 412 in real time (30 fps) in their native 
compressed fonn. In a preferred embodiment, a database is used for image storage, retrieval, 
playback and manipulation. If multiple receiver stations are coupled to the network then any 

30 station may have access to any of the other receivers live or recorded data. 

The receiver includes modules for setting alarm points and outputs at each video 
receiver. In a preferred embodiment, the receiver software is implemented in Visual Basic 
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and the underlying database is a Btrieve database engine with option to use other proprietary 
types of database engines. The receiver also includes image processing software for 
displaying multiple views of multiple cameras and a user input interface shown in figures 5 
for setting various parameters such as security, camera scheduling, reports and such like. 

Thus, it may be seen that the present invention provides a fully integrated near real 
time video surveillance, monitoring and capture system that is cost effective and highly 
flexible. Further, the mixer does not lose any frames during the actual mixing, i.e. at 30 fps. 
Other configurations of the mixer may also be implemented, such as having one or more 
mixer boards plug directly into the receiver computer without using a LAN. 

Although the invention has been described with reference to certain specific 
embodiments, various modifications thereof will be apparent to those skilled in the art 
without departing from the spirit and scope of the invention as outlined in the claims 
appended hereto. 
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THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE 
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS: 

1 . A digital video system comprising: 
5 a) a video mixer for spatially mixing a plurality of analog video input signals in 

real time and compressing said mixed signals for transmission; and 
b) a receiver for receiving and decompressing said compressed signals for 

display in near real time. 

10 2. A digital video system as defined in claim 1, said video transmitter including 

a) a video digitizer for receiving and digitizing a plurality of analog video inputs; 

b) a spatial mixer for receiving said plurality of digitized video signals and 
spatially mixing said signals in real time into a single frames of a 
predetermined n by m pixel images; and 

15 c) a video compressor for compressing each said single frame image. 

3. A digital video system as defined in claim 1 , said compression being wavelet 
compression. 

20 4. A digital video system as defined in claim 1, said video mixer including a 

network interface for transmitting said compressed signals on a LAN. 

5. A digital video system as defined in claim 1 , including a plurality of video 
mixers coupled to a network and said receiver for receiving said compressed 

25 imaged from said plurality of transmitters. 

6. A digital video system comprising: 

a) a video digitizer for receiving and digitizing a plurality of analog video 
inputs; 

30 b) a mixer for receiving said plurality of digitized video signals and spatially 

mixing said signals in real time into a single frames of a predetermined n 
by m pixel images; 
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c) a video compressor for compressing each said single frame images; 

d) a transmitter for transmitting said images; and 

e) a receiver for receiving and decompressing said transmitted images. 

7, A method for combining a plurality of video signals in a video mixer, comprising the 
steps of : 

a) digitizing each of a plurality of analog video input channels as a sequence of 
frames; 

b) storing said digitized frames for each channel; 

c) retrieving said stored frames in accordance with a predetermined spatial layout 
of said channels; and 

d) compressing each said retrieved frame to produce a single frame of said 
spatially arranged frames 
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